Journals
  Publication Years
  Keywords
Search within results Open Search
Please wait a minute...
For Selected: Toggle Thumbnails
Cross-view geo-localization method based on multi-task joint learning
Xianlan WANG, Jinkun ZHOU, Nan MU, Chen WANG
Journal of Computer Applications    2023, 43 (5): 1625-1635.   DOI: 10.11772/j.issn.1001-9081.2022040541
Abstract339)   HTML7)    PDF (3631KB)(235)       Save

Multi-task Joint Learning Model (MJLM) was proposed to solve the performance improvement bottleneck problem caused by the separation of viewpoint-invariant feature and view transformation method in the existing cross-view geo-localization methods. MJLM was made up of a proactive image generative model and a posterior image retrieval model. In the proactive generative model, firstly, Inverse Perspective Mapping (IPM) for coordinate transformation was used to explicitly bridge the spatial domain difference so that the spatial geometric features of the projected image and the real satellite image were approximately the same. Then, the proposed Cross-View Generative Adversarial Network (CVGAN) was used to match and restore the image contents and textures at a fine-grained level implicitly and synthesize smoother and more real satellite images. The posterior retrieval model was composed of Multi-view and Multi-supervision Network (MMNet), which could perform image retrieval tasks with multi-scale features and multi-supervised learning. Experimental results on Unmanned Aerial Vehicle (UAV) dataset University-1652 show that MJLM achieves the Average Precision (AP) of 89.22% and Recall (R@1) of 87.54%, respectively. Compared with LPN (Local Pattern Network) and MSBA (MultiScale Block Attention), MJLM has the R@1 improved by 15.29% and 1.07% respectively. It can be seen that MJLM processes the cross-view image synthesis and retrieval tasks together to realize the fusion of view transformation and viewpoint-invariant feature methods in an aggregation, improves the precision and robustness of cross-view geo-localization significantly and verifies the feasibility of the UAV localization.

Table and Figures | Reference | Related Articles | Metrics
Unmanned aerial vehicle image localization method based on multi-view and multi-supervision network
Jinkun ZHOU, Xianlan WANG, Nan MU, Chen WANG
Journal of Computer Applications    2022, 42 (10): 3191-3199.   DOI: 10.11772/j.issn.1001-9081.2021081518
Abstract513)   HTML18)    PDF (2090KB)(184)       Save

Aiming at the problem of low accuracy of the existing cross-view image matching algorithms, an Unmanned Aerial Vehicle (UAV) image localization method based on Multi-view and Multi-supervision Network (MMNet) was proposed. Firstly, in the proposed method, satellite perspective and UAV perspective were integrated, global and local features were learnt under a unified network architecture, then classification network was trained and metric tasks were performed in multi-supervision way. Specifically, the Reweighted Regularization Triplet loss (RRT) was mainly used by MMNet to learn global features. In this loss, the reweighting and distance regularization strategies were to solve the problems of imbalance of multi-view samples and structure disorder of the feature space. Simultaneously, in order to pay attention to the context information of the central building in target location, the local features were obtained by MMNet via square ring cutting. After that, the cross entropy loss and RRT were used to perform classification and metric tasks respectively. Finally, the global and local features were aggregated by using a weighted strategy to present target location images. MMNet achieved Recall@1 (R@1) of 83.97% and Average Precision (AP) of 86.96% in UAV localization tasks on the currently popular UAV dataset University-1652. Experimental results show that MMNet significantly improves the accuracy of cross-view image matching, and then enhances the practicability of UAV image localization compared with LCM (cross-view Matching based on Location Classification), SFPN (Salient Feature Partition Network) and other methods.

Table and Figures | Reference | Related Articles | Metrics